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DETAILED ACTION 

Response to Amendment 

1 . This Office Action is responsive to Applicant's remarks received on February 17, 
2010. Claims 1, 2, 4-6 and 8-15 remain pending. 

Claim Objections 

2. The previous claim objections have been withdrawn in light of Applicant's 
amendment. 

Claim Rejections - 35 USC §112 

3. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

4. Claims 1, 2, 4-6, and 8-15 are rejected under 35 U.S.C. 112, second paragraph, 
as being indefinite for failing to particularly point out and distinctly claim the subject 
matter which applicant regards as the invention. 
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Claims 1 and 15 require deriving a normal behavior for each position in the video 
images in terms of velocity by accumulating the data in a four-dimensional histogram 
and classifying a track as normal by comparing the frequency of occupation of a 
histogram cell with an occupancy threshold. First of all, it is unclear how the four- 
dimensional histogram is constructed. The claims state that the four dimensions are "x- 
position, y-position, x-velocity and y-velocity". As a histogram generally contains a 
vertical axis to represent the frequency of occurrence, how do the four dimensions 
relate to the vertical axis? Are the horizontal axes vectors of position and velocity? Will 
4 discrete histograms, each representing x-position, y-position, x-velocity and y-velocity 
respectively, sufficient to infringe the claim? Furthermore, the claims state that the 
comparison is done using a histogram cell "representing a corresponding position and 
velocity within the video images" with an occupancy threshold. As it is unclear how the 
histogram cell is defined, it is further unclear how a comparison using the histogram cell 
is conducted. Further clarification is required. 



Claim Rejections - 35 USC § 103 



5. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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6. Claims 1, 2, 8-12, 14 and 15 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of Owens et al. ("Application of the Self-Organising 
Map...", IEEE Article 2000, hereinafter Owens 2000) and Owens et al. ("Novelty 
Detection in Video Surveillance...", Springer Article 2002, hereinafter Owens 2002). 

Regarding claim 1, Owens 2000 discloses a method for a video motion anomaly 
detector for processing video images to detect an event of interest ("detecting 
suspicious behavior of pedestrians using a computerised grey-scale video surveillance 
system" at page 1 , section 1 , line 1 ), comprising: 

receiving, by the video motion anomaly detector, a video signal representing the 
video images to be processed ("video sequences were recorded" at page 3, section 4, 
line 4); 

extracting, by the video motion anomaly detector, at least one point feature from 
the video signal ("sequence of centroid points for each tracked object" at page 3, 
section 2, line 2); 

tracking, by the video motion anomaly detector, the position and movement of 
the at least one point feature within the video images to generate a corresponding at 
least one track, each of said at least one track representing a corresponding point 
feature in terms of its position and its velocity within each of the video images ("flow 
vector, f = [x, y, dx. dy]" at page 3, section 4, paragraph 3, line 1 ; "Each individual point 
in the trajectory is translated into a feature vector" at page 5, section 4, last sentence); 

using, by the video motion anomaly detector, an iterative learning process to 
derive a normal pattern of behavior for each track for each position within the video 
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images in terms of observed incidences of point feature velocity at said each position 
("training data consisted of 206 normal trajectories" at page 6, section 6, paragraph 2, 
line 5), wherein the learning process accumulates data representing the behavior of the 
track(s) over a period of time using four dimensions representing x-position, y-position, 
x-velocity and y-velocity, of the track(s) within the video image ("flow vector, f = [x, y, dx. 
dy]" at page 3, section 4, paragraph 3, line 1); 

comparing present behavior of a track at a certain position within the image to 
the respective derived normal pattern of behavior at the certain position in terms of 
observed point feature velocity at the certain position ("test set included 23 unusual 
trajectories and 16 normal trajectories" at page 6, section 6, paragraph 2, line 6; 
"complete trajectory was classified as unusual if it contained two or more points 
classified as unusual" at page 6, section 6, paragraph 4; "novel points are instances in 
which a trajectory segment was either in an unfamiliar orientation, or of an unfamiliar 
magnitude (i.e. unusual speed)" at page 6, section 6, paragraph 8, line 9); and 

in response to the present behavior falling outside the normal pattern of behavior 
in terms of observed point feature velocity at the certain position, the video motion 
anomaly detector generates a signal (as seen in figure 3a, points that are unusual are 
shown in black, and the trajectory is labeled as unusual). 

Owens 2000 does not explicitly disclose that the data is accumulated in a four- 
dimensional histogram and that the comparison process classifies a track according to a 
comparison of the frequency of occupation of a histogram cell representing a 
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corresponding position and velocity within the video images with an occupancy 
threshold. 

However, as the Examiner is unclear how the four-dimensional histogram of 
Applicant's claim is defined, it appears that the neural network of Owens 2000 is 
equivalent to the claimed four-dimensional histogram ("two layer neural network that is 
able to "learn" to represent distributions of the data presented to it" at page 5, section 5, 
line 2). As such, Owens 2000 also discloses that the comparison process classifies a 
track according to a comparison of the frequency of occupation of a histogram cell 
("winning neuron" at page 6, section 5, paragraph 7, line 2) representing a 
corresponding position and velocity within the video images with an occupancy 
threshold ("If this distance exceeds a threshold value then the feature vector is 
considered novel, and the trajectory is identified as suspicious" Owens 2000 at page 6, 
section 5, paragraph 7, line 5) 

Owens 2000 does not explicitly disclose that the signal is an alarm signal. 

Owens 2002 teaches a method for a video motion anomaly detector for 
processing video images to detect an event of interest (see title and figure 3), 
comprising: 

in response to the present behavior falling outside the normal pattern of behavior, 
the video motion anomaly detector generates an alarm signal ("In abnormal sequences 
the novelty accumulator nodes are allowed to increase their activity, generating an 
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alarm state" at abstract, last sentence; "alarm reporting as the event is developing" at 
page 1254, section 4, second paragraph, line 5). 

It would have obvious at the time the invention was made to one of ordinary skill 
in the art to utilize the alarm state signal of Owens 2002 to signify the unusual behavior 
of Owens 2000 such that crime prevention may occur by alerting a surveillance operator 
to the existence of possible suspicious behavior (see Owens 2002 at page 1254, 
second to last paragraph). 

Regarding claim 2, the Owens 2000 and Owens 2002 combination discloses a 
method wherein the alarm signal causes at least one of the following effects: 

draw the attention of the operator ("drawing the attention of a human operator to 
such events" Owens 2000 at page 1 , section 1 , line 6); 

place an index mark at the appropriate place in recorded video data; and 

trigger selective recording of video data. 

Regarding claim 8, the Owens 2000 and Owens 2002 combination discloses a 
method wherein the comparison process acts to classify as normal behavior a track 
adjacent or near a cell which is above the occupancy threshold, despite the track 
appearing in a cell below the occupancy threshold, where one cell is considered to be 
near another if the distance between them is below a predetermined distance threshold 
(as seen in Owens 2002 at figure 3e and 3f, these tracks are considered normal, though 
there are points that are abnormal). 

Regarding claim 9, the Owens 2000 and Owens 2002 combination discloses a 
method wherein abnormal tracks are filtered ("objective of the system is to act as a filter, 
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detecting behavior that appears out of the ordinary" Owens 2000 at page 1 , section 1 , 
line 4), whereby an active alarm signal is generated in response to an abnormal track 
which resembles a number of other abnormal tracks, in terms of at least one of position, 
velocity and time ("pedestrian moving in between parked vehicles shows high rates of 
velocity change as the path reverses direction" Owens 2000 at page 5, section 4, 
paragraph 3, line 8; a trajectory similar to the one in figure 3 would be indicated as 
unusual as it would resemble figure 3 in terms of position and velocity). 

Regarding claim 10, the Owens 2000 and Owens 2002 combination discloses a 
method wherein abnormal tracks are filtered, whereby an active alarm signal is 
generated in response only to an abnormal track which has been classified as abnormal 
on a predetermined number of occasions (i.e. once). 

Regarding claim 11, the Owens 2000 and Owens 2002 combination discloses a 
method wherein abnormal tracks are filtered, whereby an active alarm signal is 
generated in response only to a track being classified as abnormal for the first time 
(once the trajectory is indicated as unusual and the operator's attention is drawn to it, 
there is no further need to raise an alarm). 

Regarding claim 12, the Owens 2000 and Owens 2002 combination discloses a 
method wherein abnormal tracks are filtered, whereby an active alarm signal is 
generated only in response to a filtered version of a classification as abnormal rising 
above a predetermined threshold value ("If this distance exceeds a threshold value then 
the feature vector is considered novel, and the trajectory is identified as suspicious" 
Owens 2000 at page 6, section 5, paragraph 7, line 5; "complete trajectory was 
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classified as unusual if it contained two or more points classified as unusual" Owens 
2000 at page 6, section 6, paragraph 4). 

Regarding claim 14, the Owens 2000 and Owens 2002 combination does not 
explicitly disclose that subsequent active alarm signals are inhibited if caused by an 
abnormal track within a predetermined distance of another track which has previously 
generated an alarm. 

However, Owens 2000 suggests that subsequent active alarm signals can be 
inhibited if caused by an abnormal track within a predetermined distance of another 
track which has previously generated an alarm ("One of the three normal trajectories 
classified as unusual, traced a pedestrian moving. ..Upon examination of the training 
set, it was found that a trajectory in this location and direction was only represented 
once in the training set... This implies a requirement to update the neural network online 
with newly-detected normal trajectories when they occur" at page 6, section 6, 
paragraph 6; as such, subsequent unusual trajectories that resemble the initial unusual 
trajectory will not generate an alarm). 

Therefore, it would have been obvious at the time the invention was made to one 
of ordinary skill in the art to utilize the neural network updating as suggested by Owens 
2000 in the system of the Owens 2000 and Owens 2002 combination to overcome the 
problem that "novel but actually acceptable behaviour will be classified as suspicious" 
(Owens 2000 at page 6, section 6, paragraph 6, line 14). 

Regarding claim 15, Owens 2000 discloses an apparatus for processing video 
images to detect an event of interest ("standard PC" at page 7, section 7, paragraph 1, 
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last sentence; "detecting suspicious behavior of pedestrians using a computerised grey- 
scale video surveillance system" at page 1, section 1, line 1), comprising: 

a source of video images, which produces a video signal representing the video 
images to be processed ("video sequences were recorded" at page 3, section 4, line 4); 

a feature extraction device that receives the video signal, and produces data 
representing at least one point feature detected within the image ("sequence of centroid 
points for each tracked object" at page 3, section 2, line 2); 

a feature tracking device that receives the data representing said at least one 
point feature, and produces data representing a track that is representative of position 
and velocity of each of said at least one point feature within the image ("flow vector, f = 
[x, y, dx. dy]" at page 3, section 4, paragraph 3, line 1; "Each individual point in the 
trajectory is translated into a feature vector" at page 5, section 4, last sentence); 

a learning device that receives the data representing tracks of said at least one 
point feature, and derives a normal range of behavior of each position within the video 
images in terms of observed incidences of point feature velocity at said each position in 
response to operation of a learning process on the data representing the tracks 
("training data consisted of 206 normal trajectories" at page 6, section 6, paragraph 2, 
line 5); 

a classification device that receives both the signal representing the normal 
range of behavior and the data representing the tracks, and is adapted to compare the 
signal and the data for a corresponding position within the video images and to issue a 
normal/abnormal signal in accordance with the outcome of such comparison ("test set 
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included 23 unusual trajectories and 16 normal trajectories" at page 6, section 6, 
paragraph 2, line 6; "complete trajectory was classified as unusual if it contained two or 
more points classified as unusual" at page 6, section 6, paragraph 4; "novel points are 
instances in which a trajectory segment was either in an unfamiliar orientation, or of an 
unfamiliar magnitude (i.e. unusual speed)" at page 6, section 6, paragraph 8, line 9); 
and 

a generation device that receives the normal/abnormal signal and generates at 
least one active signal in response to the normal/abnormal signal indicating abnormal 
behavior of at least one track (as seen in figure 3a, points that are unusual are shown in 
black, and the trajectory is labeled as unusual). 

Owens 2000 does not explicitly disclose that the data is accumulated in a four- 
dimensional histogram and that the classification device classifies a track according to a 
comparison of the frequency of occupation of a histogram cell representing a 
corresponding position and velocity within the video images with an occupancy 
threshold. 

However, as the Examiner is unclear how the four-dimensional histogram of 
Applicant's claim is defined, it appears that the neural network of Owens 2000 is 
equivalent to the claimed four-dimensional histogram ("two layer neural network that is 
able to "learn" to represent distributions of the data presented to it" at page 5, section 5, 
line 2). As such, Owens 2000 also discloses that the classification device classifies a 
track according to a comparison of the frequency of occupation of a histogram cell 
("winning neuron" at page 6, section 5, paragraph 7, line 2) representing a 
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corresponding position and velocity within the video images with an occupancy 
threshold ("If this distance exceeds a threshold value then the feature vector is 
considered novel, and the trajectory is identified as suspicious" Owens 2000 at page 6, 
section 5, paragraph 7, line 5) 

Owens 2000 does not explicitly disclose that the signal is an alarm signal. 

Owens 2002 teaches a system for processing video images to detect an event of 
interest (see title and figure 3), comprising: 

a generation device that receives the normal/abnormal signal and generates at 
least one active signal in response to the normal/abnormal signal indicating abnormal 
behavior of at least one track ("In abnormal sequences the novelty accumulator nodes 
are allowed to increase their activity, generating an alarm state" at abstract, last 
sentence; "alarm reporting as the event is developing" at page 1254, section 4, second 
paragraph, line 5). 

It would have obvious at the time the invention was made to one of ordinary skill 
in the art to utilize the alarm state signal of Owens 2002 to signify the unusual behavior 
of Owens 2000 such that crime prevention may occur by alerting a surveillance operator 
to the existence of possible suspicious behavior (see Owens 2002 at page 1254, 
second to last paragraph). 
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7. Claim 13 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Owens 2000 and Owens 2002 as applied to claim 1 above, and further 
in view of Kamin (US 4,1 98,653). 

The Owens 2000 and Owens 2002 combination discloses the elements of claim 
1 as described in the 103 rejection above. 

The Owens 2000 and Owens 2002 combination does not disclose that 
subsequent active alarm signals are inhibited for a predetermined time interval after a 
first active alarm signal has been produced. 

Kamin discloses a method in the same field of endeavor of video tracking 
wherein subsequent active alarm signals are inhibited for a predetermined time interval 
after a first active alarm signal has been produced ("first alarm pulse A' (Fig. 3d), which 
originates from field 2 and which would normally result in a spurious alarm, is 
suppressed at the right time" at col. 4, line 24). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the delay of Kamin in the alarm generation device of the Owens 
2000 and Owens 2002 combination to provide "high sensitivity with respect to events 
which are relevant to a genuine alarm" (Kamin at col. 1 , line 59). 

8. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Owens 2000 and Owens 2002 as applied to claim 1 above, and further 
in view of Franke et al. (US 6,41 1 ,328). 

The Owens 2000 and Owens 2002 combination discloses a method wherein: 
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the learn behavior stage segregates the tracks according to a velocity threshold 
("relatively low rates of velocity change are seen in normal pedestrian motion, while a 
pedestrian moving in between parked vehicles shows high rates of velocity change" 
Owens 2000 at page 3, section 4, paragraph 3, line 6); 

data concerning the mobile tracks is stored in said four-dimensional histogram 
(see claim 1 above). 

The Owens 2000 and Owens 2002 combination does not disclose that tracks 
moving at a velocity below the velocity threshold are considered stationary while tracks 
moving at a velocity in excess of the velocity threshold are considered mobile. 

Franke et al. teaches a method for a video motion anomaly detector for 
processing video images to detect an event of interest ("visible spectrum images may 
be used to detect traffic-related incidents" at col. 11, line 14), wherein: 

tracks moving at a velocity below the velocity threshold are considered stationary 
while tracks moving at a velocity in excess of the velocity threshold are considered 
mobile ("Slow vehicles can be identified in a time sequence of images by a method 
comprising estimating vehicle speed for each vehicle as above and identifying vehicles 
moving faster than a predetermined speed. Speeding vehicles can be identified by an 
analogous method in which vehicles moving faster than a predetermined speed are 
identified in the image sequence" at col. 8, line 59). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the velocity distinction of Franke et al. to identify the tracks of the 
Owens 2000 and Owens 2002 combination such that the operator may be warned of 
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"the presence of substantially stationary objects in a relatively high-risk region of 
interest" (Franke et al. at col. 10, line 31) and the proper corrective action may occur 
thusly. 

The Owens 2000, Owens 2002 and Franke et al. combination does not explicitly 
disclose that data concerning the stationary tracks is stored in a two-dimension 
histogram representing x-position and y-position within the video image. 

However, it is well-known in the art to utilize two-dimensional histograms to store 
position data and therefore would have been obvious at the time the invention was 
made to store the stationary data of the Owens 2000, Owens 2002, and Franke et al. 
combination in a two-dimensional histogram. 

9. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Owens 2000 and Owens 2002 as applied to claim 1 above, and further 
in view of Zuniga (US 5,546,474). 

The Owens 2000 and Owens 2002 combination discloses the elements of claim 
1 as described in the 103 rejection above. 

The Owens 2000 and Owens 2002 combination does not disclose that a cell size 
of the histogram varies with speed. 

Zuniga discloses a method in the same field of endeavor of region detection 
wherein a cell size ("size of a cell can vary" at col. 10, line 14) of the histogram varies 
with speed ("moment of inertia" at col. 10, line 9). 
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It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the cell varying of Zuniga in the histogram of the Owens 2000 
and Owens 2002 combination to allow the user to evaluate a "performance/quality 
tradeoff' (Zuniga at col. 10, line 15). 

10. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Owens 2000 and Owens 2002 as applied to claim 1 above, and further 
in view of Jepson et al. (US 7,058,205). 

The Owens 2000 and Owens 2002 combination discloses the elements of claim 
1 as described in the 103 rejection above. 

The Owens 2000 and Owens 2002 combination does not disclose that the 
histogram is periodically de-weighted in order to bias the result of the learning process 
towards more recent events. 

Jepson et al. discloses a method in the same field of endeavor of motion tracking 
wherein the histogram is periodically de-weighted in order to bias the result of the 
learning process towards more recent events ("up-weight the more recent frames" at 
col. 3, line 19). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the weight varying of Jepson et al. to the histogram of the Owens 
2000 and Owens 2002 combination such that a "optimal motion estimation is achieved" 
(Jepson et al. at col. 3, line 12). 



Application/Control Number: 10/524,554 
Art Unit: 2624 



Page 17 



Response to Arguments 

Summary of Remarks (@ response page labeled 9): As "Applicants' claims use 
x-position, y-position, x-velocity and y-velocity as the four dimensions", "one of ordinary 
skill in the art would have understood the terms 'four-dimensional histogram' and a 
'histogram cell' in the context of Applicants' claims." 

Examiner's Response: The Examiner has previously stated that it is unclear how 
those four dimensions are utilized to construct the four-dimensional histogram, thereby 
making it unclear how a cell of the four-dimensional histogram is constructed. As 
Applicant has not answered the questions posed in the 1 12 2nd paragraph rejections 
above, one of ordinary skill in the art would not understand "four-dimensional histogram" 
and "histogram cell" in context of the claims. 

Summary of Remarks (@ response page labeled 9): "Applicants' claims involve 
comparison of x-position, y-position, x-velocity and y-velocity to cells of a four- 
dimensional histogram... each combination of x-position, y-position, x-velocity and y- 
velocity. Thus one of ordinary skill in the art would have understood how to perform the 
claims comparison using histogram cells." 
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Examiner's Response: The Examiner has previously stated that it is unclear how 
a cell of the four-dimensional histogram is constructed. As Applicant has not answered 
the questions posed in the 1 12 2nd paragraph rejections above, one of ordinary skill in 
the art would not understand how to perform the claimed comparison using histogram 
cells. 

Summary of Remarks (@ response page labeled 12): "Those skilled in the art 
would have recognized that a neural network, such as that disclosed by Owens 2000, is 
not a four-dimensional histogram." 

Examiner's Response: The Examiner has previously stated that it is unclear how 
the four-dimensional histogram is defined. As such, one of ordinary skill in the art could 
surmise that the construction of the neural network of Owens 2000 is equivalent to the 
claimed construction of the four-dimensional histogram as shown in the rejections. 



Conclusion 



1 1 . THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 



Application/Control Number: 10/524,554 Page 19 

Art Unit: 2624 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

12. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to KATRINA FUJITA whose telephone number is 
(571)270-1574. The examiner can normally be reached on M-Th 8-5:30pm, F 8- 
4:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Bali can be reached on (571) 272-7415. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Katrina Fujita/ 
Examiner, Art Unit 2624 



A/IKKRAM BALI/ 
Supervisory Patent Examiner, Art Unit 2624 



